Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Cache Cover Art Calls #3015

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

feat: Cache Cover Art Calls #3015

wants to merge 2 commits into from

Conversation

anshg1214
Copy link
Member

This PR Implements a client-side caching system for cover art using IndexedDB (with LocalStorage fallback) to improve performance and reduce server load. The system includes automatic cache expiration and cleanup mechanisms.

Copy link
Member

@MonkeyDo MonkeyDo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I deployed to test.LB for in-situ testing, and it doesn't seem to be working for me.
I can see the stores in IndexedDB but it does not seem to store anything, the stores are empty, and going back and forth on two pages seems to call the CAA every time for all albums.

storeName: "coverart",
});

const coverArtCacheExpiry = localforage.createInstance({
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there an advantage to having a separate cache instead of storing an object with {expiry : xxxx, data: yyyyy} ?
Is it to make the homemade garbage collection more efficient?

I also saw that there was a plugin for handling expiry, and they implemented it using the same DB instance but adding a separate key-value pair, the key being based on the original key.
That could be another way to do it: https://github.com/LuudJanssen/localforage-cache/blob/master/src/localforage.js#L117-L124

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe, one of the important factors of saving data in IndexedDB / LocalStorage is the cleanup process. What removeAllExpiredCacheEntries does is run over all the keys and check if the key has been expired or not.

Saving the expiration key as ${key}_expires_${hash} makes this process slower as we'll have to index all keys, and filter down the keys of this regex and then process.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

interface CacheEntry {
  data: string;
  expiry: number
}

This approach is definitely better because a Single operation ensures data and expiry are always in sync, and makes the code simpler. BUT, when checking if data is expired, this forces you to deserialize the entire object which will increase memory usage and processing time.

@anshg1214
Copy link
Member Author

I can see the stores in IndexedDB but it does not seem to store anything, the stores are empty

I deployed the PR back to test, and I believe the PR is working fine. Did you try refreshing the DB again?
image

and going back and forth on two pages seems to call the CAA every time for all albums.

That's strange. I checked navigating through through the pages where release cards are being rendered, things are working fine. Can you share the network logs?

@MonkeyDo
Copy link
Member

MonkeyDo commented Nov 7, 2024

Yes I had refreshed the DB, but still nothing.

This time around testing, I do see two entries (for the same cover?) in there, but the other covers are still being loaded every time.
HAR logs I will send privately as they contain cookies.

The duplicate entry with true/false at the end that I have in IndexedDB, in case it is relevant:
image

@MonkeyDo
Copy link
Member

MonkeyDo commented Nov 7, 2024

OK, so I see that the issue might be between my chair and my keyboard :)

I was looking at my dashboard when testing, I didn't realize this was going to apply only to the release card.
Re-reading your comment made me realize that.

I tested again, and I'm still not sure it's working as expected.
The SVG at the top of the artist page that loads a cover art collage makes it hard to be sure, but it still seems to hit the CAA endpoints. I tested with https://test.listenbrainz.org/artist/f58384a4-2ad2-4f24-89c5-c7b74ae1cce7/ which has many albums:

  • scrolled down and expanded to see live albums (won't be in the collage for sure)
  • scroll horizontally to load all live albums covers
  • refresh the page, wait for collage to load, purge network logs
  • scroll down and expand until I am on the live sections again
  • purge the network logs again
  • scroll the live section > 144 requests to the CAA

For context, checking the indexedDB after all that, I see 13 entries only
I am assuming I should I be seeing no requests to the CAA and more entries in indexedDB.

@@ -756,13 +757,23 @@ const getAlbumArtFromReleaseGroupMBID = async (
optionalSize?: CAAThumbnailSizes
): Promise<string | undefined> => {
try {
const cacheKey = `rag:${releaseGroupMBID}-${optionalSize}`;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rag? Release Art Group?

@@ -756,13 +757,23 @@ const getAlbumArtFromReleaseGroupMBID = async (
optionalSize?: CAAThumbnailSizes
): Promise<string | undefined> => {
try {
const cacheKey = `rag:${releaseGroupMBID}-${optionalSize}`;
const cachedCoverArt = await getCoverArtCache(cacheKey);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs to be applied to getAlbumArtFromListenMetadata somewhere as well, which is used by the ListenCard component.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants